{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "05_TensorFlow",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RvwFrkzSPbw7",
        "colab_type": "text"
      },
      "source": [
        "# TensorFlow\n",
        "\n",
        "In this notebook, we'll learn the basics of [TensorFlow + Keras](https://tensorflow.org), which is a machine learning library used to build dynamic neural networks. We'll learn about the basics, like creating and using Tensors."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0aqN-ffaP4t1",
        "colab_type": "text"
      },
      "source": [
        "<div align=\"left\">\n",
        "<a href=\"https://github.com/madewithml/basics/blob/master/notebooks/05_TensorFlow.ipynb\" role=\"button\"><img class=\"notebook-badge-image\" src=\"https://img.shields.io/static/v1?label=&amp;message=View%20On%20GitHub&amp;color=586069&amp;logo=github&amp;labelColor=2f363d\"></a>&nbsp;\n",
        "<a href=\"https://colab.research.google.com/github/madewithml/basics/blob/master/notebooks/05_TensorFlow.ipynb\"><img class=\"notebook-badge-image\" src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SFa_PSr2tvaC",
        "colab_type": "text"
      },
      "source": [
        "# Set seeds"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eLAkqoRKtyFD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%tensorflow_version 2.x\n",
        "import numpy as np\n",
        "import tensorflow as tf"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l9krh147uJOV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "SEED = 1234"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1uLEnBgft22Y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Set seed for reproducibility\n",
        "np.random.seed(seed=SEED)\n",
        "tf.random.set_seed(SEED)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "08AUKP9xu8YQ",
        "colab_type": "text"
      },
      "source": [
        "# Basics"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TQTCJ9-s7ZPg",
        "colab_type": "code",
        "outputId": "a6d18115-a588-47cf-d308-1da13c7883e7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Constants\n",
        "x = tf.constant(1)\n",
        "print (x)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(1, shape=(), dtype=int32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o3jBRfYZuNqF",
        "colab_type": "code",
        "outputId": "09b303ac-e779-49ac-e682-aa5e5b0c2a01",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Creating a random tensor\n",
        "x = tf.random.uniform((2,3))\n",
        "print(f\"Type: {x.dtype}\")\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"Values: \\n{x}\")"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Type: <dtype: 'float32'>\n",
            "Size: (2, 3)\n",
            "Values: \n",
            "[[0.5380393  0.36461866 0.5816301 ]\n",
            " [0.24382842 0.43033564 0.90619123]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pho5A7JluNvj",
        "colab_type": "code",
        "outputId": "a001d00b-676b-4b87-9fe3-494579934b85",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "# Zero and Ones tensor\n",
        "x = tf.zeros((2, 3))\n",
        "print (x)\n",
        "x = tf.ones((2, 3))\n",
        "print (x)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(\n",
            "[[0. 0. 0.]\n",
            " [0. 0. 0.]], shape=(2, 3), dtype=float32)\n",
            "tf.Tensor(\n",
            "[[1. 1. 1.]\n",
            " [1. 1. 1.]], shape=(2, 3), dtype=float32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UTecl1RduNtL",
        "colab_type": "code",
        "outputId": "e0b67607-5b1d-4a88-9628-cba09bf97705",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# List → Tensor\n",
        "x = tf.convert_to_tensor([[1, 2, 3],[4, 5, 6]], dtype='int32')\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"Values: \\n{x}\")"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: (2, 3)\n",
            "Values: \n",
            "[[1 2 3]\n",
            " [4 5 6]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2OQTnxWOuNnY",
        "colab_type": "code",
        "outputId": "64902a24-3db0-4dfc-c94c-166413588638",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# NumPy array → Tensor\n",
        "x = tf.convert_to_tensor(np.random.rand(2, 3), dtype='float32')\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"Values: \\n{x}\")"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: (2, 3)\n",
            "Values: \n",
            "[[0.19151945 0.62210876 0.43772775]\n",
            " [0.7853586  0.77997583 0.2725926 ]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8K2kWrkZuilf",
        "colab_type": "code",
        "outputId": "bdda0aff-9731-4a08-b8db-0ee325a6ea2f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "# Changing tensor type\n",
        "x = tf.random.uniform((2,3))\n",
        "print(f\"Type: {x.dtype}\")\n",
        "x = tf.random.uniform((2,3), dtype='float64')\n",
        "print(f\"Type: {x.dtype}\")"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Type: <dtype: 'float32'>\n",
            "Type: <dtype: 'float64'>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6LxCmxqFu6sq",
        "colab_type": "text"
      },
      "source": [
        "# Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yfYLm_1Buixy",
        "colab_type": "code",
        "outputId": "e99fd1d3-f2e7-4e45-9b90-20582d03e8cc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Addition\n",
        "x = tf.random.uniform((2,3))\n",
        "y = tf.random.uniform((2,3))\n",
        "z = x + y\n",
        "print(f\"Size: {z.shape}\")\n",
        "print(f\"Values: \\n{z}\")"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: (2, 3)\n",
            "Values: \n",
            "[[0.9140382 1.0755982 0.8527149]\n",
            " [0.4672935 0.3658085 1.2775037]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "22abfI18uiuw",
        "colab_type": "code",
        "outputId": "4a146149-6460-42ee-f5d8-948c4163d3f7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Dot product\n",
        "x = tf.random.uniform((2,3))\n",
        "y = tf.random.uniform((3,2))\n",
        "z = tf.matmul(x, y)\n",
        "print(f\"Size: {z.shape}\")\n",
        "print(f\"Values: \\n{z}\")"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: (2, 2)\n",
            "Values: \n",
            "[[1.1487825 1.2686723]\n",
            " [0.8971158 0.6762365]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p1ztJNrruiqv",
        "colab_type": "code",
        "outputId": "9ea965f7-b7aa-4298-fc6f-0bdc77e1621f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "source": [
        "# Transpose\n",
        "x = tf.random.uniform((2,3))\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"Values: \\n{x}\")\n",
        "y = tf.transpose(x)\n",
        "print(f\"Size: {y.shape}\")\n",
        "print(f\"Values: \\n{y}\")"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: (2, 3)\n",
            "Values: \n",
            "[[0.2962978  0.6645019  0.5157938 ]\n",
            " [0.16170895 0.2753886  0.07371306]]\n",
            "Size: (3, 2)\n",
            "Values: \n",
            "[[0.2962978  0.16170895]\n",
            " [0.6645019  0.2753886 ]\n",
            " [0.5157938  0.07371306]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zoLDryFYuioF",
        "colab_type": "code",
        "outputId": "dec679f5-b313-4f59-ea31-1b0d72246898",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "# Reshape\n",
        "x = tf.random.uniform((2,3))\n",
        "z = tf.reshape(x, (1, 6))\n",
        "print(f\"Size: {z.shape}\")\n",
        "print(f\"Values: \\n{z}\")"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: (1, 6)\n",
            "Values: \n",
            "[[0.5523262  0.9752618  0.6437291  0.07045114 0.5783918  0.0563972 ]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HcW6i9xJwU2Q",
        "colab_type": "code",
        "outputId": "04675980-4220-4254-ff00-6954d6238c67",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "# Dimensional operations\n",
        "x = tf.random.uniform((2,3))\n",
        "print(f\"Values: \\n{x}\")\n",
        "y = tf.math.reduce_sum(x, axis=0) # add each row's value for every column\n",
        "print(f\"Values: \\n{y}\")\n",
        "z = tf.math.reduce_sum(x, axis=1) # add each columns's value for every row\n",
        "print(f\"Values: \\n{z}\")"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Values: \n",
            "[[0.45171046 0.9757304  0.96276057]\n",
            " [0.49976635 0.46484458 0.65036225]]\n",
            "Values: \n",
            "[0.9514768 1.440575  1.6131228]\n",
            "Values: \n",
            "[2.3902016 1.6149732]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kqxljkudzH0M",
        "colab_type": "text"
      },
      "source": [
        "# Indexing, Splicing and Joining"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q8-w1Cb3wsj0",
        "colab_type": "code",
        "outputId": "6261c40f-e41b-435f-81f2-5f34d01db1af",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "x = tf.random.uniform((2,3))\n",
        "print (f\"x: \\n{x}\")\n",
        "print (f\"x[:1]: \\n{x[:1]}\")\n",
        "print (f\"x[:1, 1:3]: \\n{x[:1, 1:3]}\")"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x: \n",
            "[[0.80235374 0.6862793  0.35347772]\n",
            " [0.5970619  0.8998532  0.05612409]]\n",
            "x[:1]: \n",
            "[[0.80235374 0.6862793  0.35347772]]\n",
            "x[:1, 1:3]: \n",
            "[[0.6862793  0.35347772]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jBGk_740wsm3",
        "colab_type": "code",
        "outputId": "b72465d4-2850-49dc-a3e7-d2de2a024519",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "# Select with dimensional indicies\n",
        "x = tf.random.uniform((2,3))\n",
        "print(f\"Values: \\n{x}\")\n",
        "\n",
        "col_indices = tf.convert_to_tensor([0, 2])\n",
        "chosen = tf.gather(x, indices=col_indices, axis=1) # values from column 0 & 2\n",
        "print(f\"Values: \\n{chosen}\") "
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Values: \n",
            "[[0.13242006 0.8582847  0.50360227]\n",
            " [0.7038616  0.42338693 0.7614349 ]]\n",
            "Values: \n",
            "[[0.13242006 0.50360227]\n",
            " [0.7038616  0.7614349 ]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UI_hboLNwsqQ",
        "colab_type": "code",
        "outputId": "ac5db769-f6b7-4388-c1ff-484a22266d8a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "# Concatenation\n",
        "x = tf.random.uniform((2,3))\n",
        "print(f\"Values: \\n{x}\")\n",
        "y = tf.concat([x, x], axis=0) # stack by rows (axis=1 to stack by columns)\n",
        "print(f\"Values: \\n{y}\")"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Values: \n",
            "[[0.46948004 0.64634395 0.07618201]\n",
            " [0.39206338 0.23859632 0.16841686]]\n",
            "Values: \n",
            "[[0.46948004 0.64634395 0.07618201]\n",
            " [0.39206338 0.23859632 0.16841686]\n",
            " [0.46948004 0.64634395 0.07618201]\n",
            " [0.39206338 0.23859632 0.16841686]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lK1OQUYL1bE3",
        "colab_type": "text"
      },
      "source": [
        "# Gradients"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Ft6PAeW0WCe",
        "colab_type": "code",
        "outputId": "961fa26f-ed3a-49fb-bcb8-dc1511fc1ff0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Tensors with gradient bookkeeping\n",
        "x = tf.constant(3.0)\n",
        "with tf.GradientTape() as g:\n",
        "  g.watch(x)\n",
        "  y = 3 * x + 2\n",
        "g.gradient(y, x).numpy()"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VF5Q5kfs1rXZ",
        "colab_type": "text"
      },
      "source": [
        "* $ y = 3x + 2 $\n",
        "* $ \\frac{\\partial(y)}{\\partial(x)} = 3 $"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kseQSKj72H8S",
        "colab_type": "text"
      },
      "source": [
        "# GPUs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZE-ZyECv0WOX",
        "colab_type": "code",
        "outputId": "df8ae7da-7547-425c-bf35-3423c38ee1f8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Is CUDA available?\n",
        "print (tf.config.list_physical_devices('GPU'))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n5sWo3Yv2MxO",
        "colab_type": "text"
      },
      "source": [
        "If False (CUDA is not available), let's change that by following these steps: Go to *Runtime* > *Change runtime type* > Change *Hardware accelertor* to *GPU* > Click *Save*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ewamITzX2W-B",
        "colab_type": "code",
        "outputId": "81f1cb48-fd61-4f82-c2bb-4edc8d9becd3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "%tensorflow_version 2.x\n",
        "import tensorflow as tf"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "TensorFlow 2.x selected.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IwsrvGad2NDO",
        "colab_type": "code",
        "outputId": "d2fb48e3-b137-4523-dd3b-43208c555b55",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Is CUDA available now?\n",
        "print (tf.config.list_physical_devices('GPU'))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xr1Vsnq7CLpB",
        "colab_type": "text"
      },
      "source": [
        "---\n",
        "Share and discover ML projects at <a href=\"https://madewithml.com/\">Made With ML</a>.\n",
        "\n",
        "<div align=\"left\">\n",
        "<a class=\"ai-header-badge\" target=\"_blank\" href=\"https://github.com/madewithml/basics\"><img src=\"https://img.shields.io/github/stars/madewithml/basics.svg?style=social&label=Star\"></a>&nbsp;\n",
        "<a class=\"ai-header-badge\" target=\"_blank\" href=\"https://www.linkedin.com/company/madewithml\"><img src=\"https://img.shields.io/badge/style--5eba00.svg?label=LinkedIn&logo=linkedin&style=social\"></a>&nbsp;\n",
        "<a class=\"ai-header-badge\" target=\"_blank\" href=\"https://twitter.com/madewithml\"><img src=\"https://img.shields.io/twitter/follow/madewithml.svg?label=Follow&style=social\"></a>\n",
        "</div>\n",
        "             "
      ]
    }
  ]
}